The present invention generally relates to the field of capturing facial expressions of users wearing a head-mounted display (HMD).
The recent introduction of consumer-level HMDs has led to a revival in virtual reality and is drawing wide interest from consumers for gaming and online virtual worlds. With the help of existing motion capture and hand tracking technologies, users can navigate and perform actions in fully immersive virtual environments. But users lack a technological solution for face-to-face communication that conveys compelling facial expressions and emotions in virtual environments.
Facial animation has been mostly dominated by optical capture systems that use cameras or depth sensors. Methods for facial representations, tracking, mapping, and animation have been developed, greatly impacting film and game production. However, because a typical face is more than 60% occluded by a HMD, established optical sensing methods that achieve the desired facial tracking quality fail to capture nearly the entire upper face.